The invention is generally directed to clustered computer systems, and in particular, to managing ownership of I/O bridge devices therein.
xe2x80x9cClusteringxe2x80x9d generally refers to a computer system organization where multiple computers, or nodes, are networked together to cooperatively perform computer tasks. An important aspect of a computer cluster is that all of the nodes in the cluster present a single system imagexe2x80x94that is, from the perspective of a user, the nodes in a cluster appear collectively as a single computer, or entity.
Clustering is often used in relatively large multi-user computer systems where high performance and reliability are of concern. For example, clustering may be used to provide redundancy, or fault tolerance, so that, should any node in a cluster fail, the operations previously performed by that node will be handled by other nodes in the cluster. Clustering is also used to increase overall performance, since multiple nodes can often handle a larger number of tasks in parallel than a single computer otherwise could. Often, load balancing can also be used to ensure that tasks are distributed fairly among nodes to prevent individual nodes from becoming overloaded and therefore maximize overall system performance. One specific application of clustering, for example, is in providing multi-user access to a shared resource such as a database or a storage device, since multiple nodes can handle a comparatively large number of user access requests, and since the shared resource is typically still available to users even upon the failure of any given node in the cluster.
A clustered computer system, like many other types of computer systems, often requires external communication with various resources, referred to hereinafter as xe2x80x9cInput/Outputxe2x80x9d (I/O) resources, e.g., workstations, network adaptors, storage controllers, etc. In many such systems, I/O resources are coupled to a clustered computer system using one or more I/O bridge devices, which serve the function of xe2x80x9cbridgingxe2x80x9d between two network fabrics, e.g., an I/O expansion bus and a peripheral bus such as a Peripheral Component Interconnect (PCI) bus. I/O bridge devices may be mounted within the same physical enclosure as a node in a clustered computer system, or may be disposed in a separate housing altogether.
In many clustered computer systems, bridge devices are configurable and required to be managed by one of the nodes. Often, it is a requirement that each bridge device be managed by exactly one node at any given point in time. As such, failure of a node in a clustered computer system may result in the loss of communication with any bridge devices owned by that node.
Given the desirability of maximizing availability in clustered computer systems, it is often desirable to maintain availability to all bridge devices and resources thereunder,xe2x80x94even in the event of node failures. As such, it would be highly desirable to facilitate the change of ownership of a bridge device from a failing node to another node that is still functioning. The functioning node could then assume management of the bridge device and allow access to those resources under the bridge device.
Switching ownership of a bridge device is not, however, a trivial process when multiple independent nodes are actively participating in clustering operations. A bridge device typically must be managed by only one node at any point in time due to the fact that certain facilities in the bridge device, e.g, the bridge network address, the address mapping of register space in the bridge device, the address mapping of resources under the bridge device and the power management of the bridge device, must be specifically configured to work with the owning node. However, the proper operation of a bridge device may be compromised when its ownership is being transferred from one node to another if an original owner attempts to access the bridge device during the ownership change. In a clustered computer system, this is typically a significant concern because it is difficult, if not impossible, for an independent node to tell if another node has completely failed or is just responding slowly. Further, if a failed node recovers, a risk exists that the recovered node may attempt to manage bridge devices it owned prior to the failure, but which were switched to a new owner at the time of the failure. Doing so could corrupt the bridge device and cause a partial or complete system failure.
Therefore, a need exists in the art for a manner of changing ownership of a bridge device in a clustered computer system, in particular, to permit reliable and efficient changeover of ownership with minimal interruption in system availability.
The invention addresses these and other problems associated with the prior art by providing a clustered computer system, bridge device and method including support for an atomic ownership change operation that ensures orderly and reliable ownership management of an Input/Output (I/O) bridge device. In particular, a lock indicator is associated with a bridge device, and is used to ensure that only one node is capable of claiming ownership of the bridge device at a time. Moreover, the ability to set one or more configuration parameters for the bridge device is restricted only to a node having ownership of the bridge device, so that an orderly change in ownership can be performed.
In the illustrated embodiments, a lock indicator is utilized to indicate a xe2x80x9clockedxe2x80x9d or xe2x80x9cunlockedxe2x80x9d status of the bridge device. Whenever the lock indicator register indicates that the bridge device is unlocked, an atomic operation such as an access request to a lock indicator register may be utilized to both set the indicator to indicate a locked status for the bridge device, and to associate the bridge device with a source node that initiated the atomic operation. Moreover, in connection with the lock indicator, write access to one or more configuration parameter registers may be controlled such that only that node that is associated with the bridge device is permitted to update such configuration parameter registers.
These and other advantages and features, which characterize the invention, are set forth in the claims annexed hereto and forming a further part hereof. However, for a better understanding of the invention, and of the advantages and objectives attained through its use, reference should be made to the Drawings, and to the accompanying descriptive matter, in which there is described exemplary embodiments of the invention.